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\ 

Abstract 

A software usage models describes the prospective use of a program in its intended 
environment and allows the generation of random test cases leading to unbiased esti- 
mates of the failure risk, i.e., the expected loss by program failure. We concentrate on 
usage models of Mau-kov type and show that by suitable changes of the probabilities of 
state transitions during test, the precision of the risk estimate can be optimized. An 
algorithm for the computation of optimal transition probabilities is presented, and 
experimental results based on a C++ implementation of this algorithm are reported. 

1 Introduction 

Recently, software usage models of Markov type have found considerable in- 
terest (see [10, 11, 1*2, 9|). The purpose of a software usage nnodel is to give 
a formal description of the expected operational use of a software system, i.e., 
its use in its intended application environment. Such a model is an essential 
prerequisite for statistical testing, a special variant of random testing allowing 
predictions on the operational behavior of the software ([8, 9]). 

Markov software usage models aim at representing the (estimated) distri- 
bution of possible uses by means of a Markov chain. This probabilistic concept 
combines the ability of picturing relative complex dynamic use structures with 
the advantage of still being mathematically tractable. A Markov usage model 
is based on a directed graph G — (V, A), where 

• is a set of nodes, representing usage states (e.g., program invocation, 
program termination, input/output screens), and 

• -4 is a set of arcs, representing state transitions which always correspond 
to specific operations of the program. An arc from state i to state j can 
be denoted by the ordered pair (i, j). 

Furthermore, each arc is labelled by a transition probability p{i, j). This 

value indicates the relative frequency of a transition to state j, given that the 
current slate is state i, during the operational use of the program. 
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Abstract 

A software usage models describes the prospective use of a program in its intended 
environment and aUows the generation of random test cases leading to unbiased esti- 
mates of the failure risk, i.e., the expected loss by prxjgram failure: We concentrate on 
usage models of Markov type and show that by suitable changes of the probabilities of 
state transitions during test, the precision of the risk estimate can be optimized. An 
algorithm for the computation of optimal transition probabilities is presented, and 
experimental results based on a C-f+ implementation of this algorithm are reported. 

1 Introduction 

Recently, software usage models of Markov type have found considerable in- 
terest (see [10, 11, 12, 9]). The purpose of a software usage model is to give 
a formal description of the expected operational use of a software system i e 
Its use in Its intended application environment. Such a model is an essential 
prerequisite for statistical testing, a special variant of random testing allowing 
predictions on the operational behavior of the software ([8, 9]). 

Markov software usage models aim at representing the (estimated) distri- 
bution of possible uses by means of a Markov chain. This probabilistic concept 
combines the ability of picturing relative complex dynamic use structures with 
the advantage of still being mathematically tractable. A Markov usage model 
is based on a directed graph G = (V, A), where 

• is a set of nodes, representing usage states (e.g., program invocation, 
program termination, input/output screens), and 

• ^ is a set of arcs, representing state transitions which always correspond 
to specific operations of the program. An arc from state i to state j can 
be denoted by the ordered pair 

Furthermore, each arc is labelled by a transition prr^bability p(i,j) This 
value indicates the relative frequency of a transition to state j, given that the 
current state is state i, during the operational use of the program 
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Fig. 1 shows a (very simple) example of a Markov usage «»°^el: A program 
has a main menu, from which the user may select two special functions The 
probabilities for the selection of function 1 resp. function 2 are estimated to be 
0 6 resp. 0.3. After the execution of the selected function, the program returns 
to the main menu. In about 10 percent of all cases, the user decides not to 
make a (new) function call, but to terminate the program 

A Markov usage model of more realistic, but still moderate complexity is 
shown in Fig. 2. It is models a (small) part of a train schedule program of the 
Austrian Federal Railways. 



0.6 



1.0 

requeat of fiaictioix 1 



-» | requeatof fiBiction2 



1.0 



tetmiBaiiMi I 



0.1 



Fig 1 A simple Markov usage model. The values assigned to the arcs are 
transition probabilities; the arcs themselves are operations of the program. 
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Fig. 2. A Markov usage model with n = 12 nodes. 

We always assume that the states (nodes) of G are numbered by 1, .... n. 
State 1 is the initial state corresponding to program invocation, state n is 
the final state, corresponding to program termination. In order to obtam a 
complete Markov chain, we add an. arc with assigned probability one from 
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node n to node n; by this self-loop, n gets an absorbing state which cannot be 
left anymore. 

Since program failures cannot be completely excluded, each program has 
a failure risk, defined as the expected loss by program failure during a single 
program execution in the operational environment (see [1, 6]). A straightfor- 
ward method for getting an unbiased estimate of the risk consists in statistical 
testing with an input distribution corresponding to the operational use, and to 
take the average loss occured during test as a risk estimate. On the base of a 
Markov usage model, a statistical test case is obtained as a state sequence or 
path (Xo, . ..,Xg) of the usage chain, beginning at the initial state and ending 
at the final state, where each transition {Xk-i,Xk) is selected according to the 
operational transition probabilities p(i,i). 

Walton, Poore and Trammell [9] have outlined a serious drawback of this 
method: There can be critical operations (connected with high failure proba- 
bility and/or high loss in case of failure) that are nevertheless invocated infre- 
quently during the operational use of the program. Then, in a Markov chain 
model, the corresponding arcs get only small probabilities of being activated, 
which may exclude these operations from test. Walton et al. suggest an adjust- 
ment of the usage distribution to ensure that such operations are tested more 
frequently, but they admit that this approach limits statistical inferences on 
the program behavior in its operational environment. 

The intention of this paper is to develop a technique of changing the tran- 
sition probabilities and compensating the eff'ect of this change in such a way 
that 

• critical operations are tested with sufficiently high frequency, and 

• the risk estimate derived from the test remains unbiased. 

To optimize the benefit of our technique, we try to find that probability change 
that allows the most precise risk estimation, (A similar approach for the more 
special case of a program with n alternative program functions has been devel- 
oped in [2].) . 

Let us emphasize that the drawback of conventional Markov usage models 
mentioned in [9] is especially prohibitive in the case of safety-critical software, 
where there are always operations with low use frequency and high criticality! 
So, without a technique overcoming this drawback, the applicability of Markov 
usage models to this type of software would be very limited. The technique 
outlined here provides a satisfactory solution to this problem. 



2 Change of transition probabilities 

Our method relies on the importance sampling technique of Rare Event Sim- 
ulation, which prescribes that a probility change in a simulation has to be 
compensated by adjusting suitable weights to the outcomes of the simulation 
run. For a Markov usage model, one obtains the following compensation for 
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a shift of the transition probabilities from p(i, j) to t(ij): Let loss(X) de- 
note the loss occuring in a single test execution along path X = (Xo, . . . , X,), 
resp. loss (A') := 0 if no failure occurs along X. Then (cf., e.g., [4]), 



(1) 



is an unbiased estimate of the rbk. This can be seen as follows. Let P resp. T 
stand for the operational distribution, resp. the test distribution obtained by 
the probability change. By Ep and Et we denote the mathematical expectation 
under distribution P and T, respectively. Obviously, 

p{X) :=l[p{Xk.uXk) resp. t{X) :=l[t{Xk-i, Xk) 

k * 

is the probability that path X occurs, given that the states are selected ac- 
cording to the probabilities p{i,j) resp. t{i,j). So for the expected value of 5 
under distribution T, 

Er(S) = (fin : loss (X)) = ^ *(»^) ' ^ 
= J2p(z) loss (x) = Ep{\oss {X)) 

X 

holds. The rightmost expression, however, is just the risk. Let us mention that 
in importance sampling, the weight p{X)/t{X) is usually called the likelihood 
ratio. 

Taking the average value of S for all random test cases (selected according 
to the changed transition probabilities) yields the overall risk estimate. 

For example, the transition probabilities in Fig. 1 could be changed to, say, 

*(menu, req-function.l) := 0.2, ^(menu, req_functionJ2) := 0.8 

(other probabihties as before). Assume that, selecting states according to these 
new probabilities, we obtain the path 

X = (invoc, menu, req-function-2, menu, reqJunction.l, menu, term), 

and that an execution of the program with inputs driving it along X reveals a 
program failure of a severity estimated by 10 cost units. Then this loss has to 
be weighted by 

0.8 0.2 

such that we derive a risk estimate of 5 = 1.125 • 10 = 11.25 from this test 
case. Of course, the risk estimate corresponding to a test case is zero, if the 
test case does not reveal a failure. If N test cases are selected, executed, and 
evaluated in the way described above, yielding risk estimates 5(1), . . . , 5(A/'), 
then the overall risk estimate is given by the arithmetic mean S of the values 
5(l),...,5(iV). 
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3 Optimizing the precision of the risk estimate 

Now let us txirn to the question how to choose the transition probabilities in 
order to get a risk estimate with maxiinum precision. As a measure for the 
unprecision of an (unbiased) estimator we take its variance, as customary in 
software reliability (cf. (7]). The variance varr(5) of 5 for test cases selected 
according to test distribution T would satisfy our requirements on a crite- 
rion; however, varr{S) cannot be determined in advance, since 5 depends on 
loss (A'), and loss {X) is unknown before the test. 

Therefore, a Bayesian approach is applied: For each fixed r, loss(x) is con- 
ceived as a random variable, whose distribution reflects the prior information 
of the tester on failure probabilities and amount of loss in case of failure. To be 
more explicit: It is assumed that the operation corresponding to arc (z, j) has a 
prior probability /(:, j) of failing, and causes a loss of /{i, j) in case of the fail- 
ure event. Failures of different arcs (i, j) occur independently from each other. 
The values /(i, j) and /(i, j) have to be estimated for each arc. Although this 
model is only an approximation to reality, it already allows the test designer 
to take account of critical operations, which is not possible in the conventional 
Markov usage model framework. 

Denoting the expectation with respect to the (estimated) prior loss distri- 
bution by Eq, the aim is to minimize EqivarriS)) by an appropriate choice 
of the test distribution T. (Notice that Eq acts, for fixed path on the loss, 
while Ep and Er act on the path.) 

In the sequel, we assume that the loss occuring at a specific program exe- 
cution mainly depends on the path x triggered by the input data. Then, for 
fixed r, loss (r) can be considered (in a first approximation) as independent 
of the actual input causing the execution of x. Of course, this is a simplifica- 
tion: For example, it may happen that input.l and input_2 lead to the same 
path X, but input-1 is processed correctly, while input,2 is not. Nevertheless, a 
Markov usage model with a sufficiently high granularity (cf. [9]) has the prop- 
erty that inputs triggering the same path are processed in a very similar way, 
such that they can be considered as "near-to-homogenous" in their failure be- 
havior. (For the notion of homogenous sets of inputs, see [3]). We emphasize 
that this assumption is not required for the unbiasedness of the estimator (1). 

By ^(x), the set of transitions (i,j) contained in path x is denoted. Ac- 
cording to the model described above, we obtain now 



loss(x)= J2 fail(f,j)/(i,j), 



(2) 



where 



fail(i,j) = / if transition (2,j) fails, 
' I 0, else, 

such that £:Q(fail(i,i)) = /(z,j). Since 

varriS) = Et{S^) - [£:t(5)J2 = Et{S') - (risk)^ 
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minimization of EQ{varT{S)) with respect to T is equivalent to minimi2ation 
of EQ{EriS^))- By a short calculation, one finds 

EQ{Er{.r-)) = Ep .£<3([loss(Xr)) . (3) 

Furthermore, because of (2), Eq( [1oss(X)]2) is equal to 

Eq I J^faiUi.i) + 53 fail(/:,m) l{i,j)l{k,m) 

= E/('«j)t^<*''-')l'+ ^ f{i,j)f{k,m)l{i,j)l{k,m), (4) 

all sums being only over arcs contained in A(X). On the usual assumption that ' 
prior failure probabilities are small quantities (only programs that are already 
relatively stable are worthwile to be exposed to tests for risk estimation!), the 
second sum on the rightrriost side of (4) is negligible, compared to the first 
sum. Omission of this second sum, insertion into (3) and re-insertiqn of the 
expressions for p(X) and t{X) leads to the following stochastic optimization 
problem: 

Minimize G{T) := eJ l[ " E ] " 

\ 

Therein, the variables t{i,j) (1 < », J < n) to be optimized have to satisfy the , 
constraint of being transition probabilities of a Markov chain with absorbmg j 
state n. Additionally, the constraint t{ij) = 0 if and only if p(i,i) = 0 must 
be satisfied, otherwise the likelihood ratio could become infinite. • 
It can be shown that the function G{T) in (5) is convex in the variables ; 
t{i,j). This fact allows an efficient solution of (5) by well-known stochastic , 
optimization techniques. ', 

4 Numerical computation of the optimal tran- | 
sition probabilities j 

For the numerical solution of (5), we use, in our implementation, an optimiza- |( 
tion algorithm of Frank-Wblfe-type (see [5], section 10.10.3). Basically, we ^ 
proceed as follows: A sample of R independent paths according to distribution 
P is drawn, and the minimization of the expected value of the product in (5) 
is replaced by the minimization of the average value of the product over the 
paths in the sample. This is done iteratively. We start with the [n x n]-matrix 
T of the variables := p{i,j) as the initial solution. The solution is im- 

proved successively: For each fixed line i of the current matrix T, consider the 
partial derivatives D{i,j) of G{T) to the variables t{ij) ( ; = 1, . . . , n). Take 



NOV 10 2003 18:50 FR CISTI ICIST 



613 952 9303 TO 17033052763 



P. 09/12 



nization 



(3) 



,711) 



(4) 

tion that 
J already 
oq!), the 
the first 
)n of the 
mization 



• (5) 



itisfy the 
ibsorbing 
= 0 must 

variables 
.tochastic 



tran- 



optimiza- 
ically, we 
5tribution 
uct in (5) 
. over the 
n]-matrix 
ion is im- 
nsider the 
,n). Take 



189 

that J = j' for which D{iJ) is minimal, and modify the current solution T by 
augmenting the value of t{ij*) by a certain stepsize. The partial derivatives 
can be computed explicitly. We use the result of this computation in 
the following, more detailed description of the algorithm: 

procedure optimize-test (P, F, L) 
A ^ = {Pihi)). F = (/(,-,;)), L ^ V 
{ for r := 1 to /? 
{ draw path X according to P\ 

compute y(r) := m>i ^0 E(.j)€>4(X) /(^i) 
for each (i, j) 

compute M{r,iJ) := number of transitions (Xk-iyXk) = } 
set T := P; 
for 5 := 1 to 5 
for i := 1 to n 
{ for each j with p(x, j) > 0 

compute D{i,j) := arithmetic mean over r = 1, . . . , ii of 
" yir) Uk^m («(A:,m))-^(->.-) M(rJJ)/t{iJ); 
determine that j = j* for which D{i,j) is minimal; 
for each j with p{ij) > 0 

1/(25)) f(,-,y); 

if U = n t{ij) := t{i.j) + l/{2s); } } 
return T; } 

By some minor modifications, both the runtime eflScency and the solution 
quality of this algorithm can still be improved. 

5 Experimental results 

Forjudging the solution quality of the approximation algorithm optimize.test 
described in the previous section, one may start with the small example of a 
Markov usage model in [9], p. 102, since it is stiU possible to calculate the 
solutions of (5) exactly Jot this case. The Markov chain in the example has 
four states (invocation, main-menu, display, and termination). The operational 
transition probabilities, the failure probabilities and the failure severities (losses 
in case of failure) are estimated as follows: 
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Because of the condition t{i,j) = 0 <»• piij) = 0, the test transition proba- 
bilities are identical to the values p{i, j). except <(2,3) =: t and ^(2,4) = 1 - <. 
Explicit solution of (5) yields t' = 0.620 as the optimal value for i. 

Our C++ implementation of optimize-test finds for /? = 100 and 5 = 
100 an approximation to the exact value of f that is is sufficiently good for 
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K 
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Runtime (in sec) 


100 


100 
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0.635 
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1000 


100 
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0.617 


59 


.100 


100 


10 


0.621 


56 



Table L Application of optimize.test to the 4-state Markov usage model: 
results and runtimes on a PC with Intel 80486 DX processor. 



practical purposes (see line 1 of Table 1). Of course, increasing the sample 
size R improves the solution quality (see line 2 of Table 1).. Note, however, 
that storing the entries M{rJJ) requires space of order 0{R • n^), which can 
be prohibitive for large Markov usage models. So we have investigated an 
alternative: Instead of generating a very large sample of paths and storing 
the quantities Y{r) and M(r, ij) for all these paths, we generate only a more 
liriiited sample, perform the approximation steps of optimize.test, and repeat 
this process K times, such that newly drawn samples can influence the current 
approximate solution. In order to obtain convergence of the algorithm, the 
stepsize for the modification of the t{ij) has to be decreased in the successive 
iterations A: = 1,...,A'. We have chosen a stepsize of l/{2sk) in the fcth 
iteration. The effect of this procedure is here even more convincing than that 
of working with a large sample as a whole (see line 3 of Table 1). However, the 
last observation cannot be generalized: for other values R and 5, we have also 
found cases where augmenting A' by a certain factor produced slightly worse 
results than augmenting R by the same factor. 

Another situation where exact solution values can be determined is the case 
of Markov chains consisting of edge-disjoint paths from node 1 to node 7i. Here, 
Theorem 2 in [2] yields an explicit formula for the optimal test probabilities. We 
have tested some examples of this type and found a good agreement between 
theoretical and experimental values. 

Finally, we present the outcomes of optimize.test for the Markov usage 
model of Fig. 2. The values f(ij) and have been estimated as follows: 
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Therein the constants / (low), m (medium) and h (high) have the following 

h 100. We have assumed that arc (8.10) corresponds to a newly implemented 
{i.e., error-prone) function with high criticality. Here, explicit solutions of (5) 
7*^^*^^f f °y™Of . so we cannot compare the output of optimize-test 

Tftl. io« ^''^ ^ = 100' ^ = and K = 10, we obtained, 

atter 196 seconds of computation time, the foUowing output matrix T- 
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.ff ^^'"P/""?*!^^^^'*^^ HiJ) with the vkluesp(z,y) unveils that the intended 
effec of favormg the test of critical operations has been achieved: While for 
say, 100 test cases m total, the generation of test sequences according to P gives 
It^ n ^ ^"'^ * probability of being tested (note that 

aireaxly (3,8) has a small probability!), transition (8.10) obtains a fair chance 
in a test sequence generation according to T. 

\in.f^T- ""^"^^f implementation, the runtime of optimize.test grows only 
usa.e ^nH ^'^«<°""^ber of entries) of the input matrices, even large Markov 
usage models can be treated in reasonable time. The bottleneck is the storage 
req_t of order O(Rn^): For large „. R has to be chosen relative y sm^f, 

inc eLin T f " '^^'^ P^^^'^''^ compensated by 

increasmg the value K. In any case, the number R • A' of generated test 

executed. Otherwise, it may happen that infrequently used, but error-prone 
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paths do not occur in the sample and get therefore no opportunity to shift the 
numbers t(i,j) to more favorable values. 

An approach for the solution of the last-mentioned problem is to re- 
formulate (5) in such a way that instead of Ep, the expected value Ep with re- 
spect to another distribution P giving a// possible state transitions fair chances 
is used. For the sake of brevity, we omit the details of this more sophisticated 
technique. 
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